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ABSTRACT 

This document represents the final report for an 
evaluation study of the Delaware Computer Inservice Program. The 
purpose of this program was to increase the use of computers in 
traditional subject areas. The program was pilot tested in three 
schools, each from a different district in Delaware, during the 
1987-88 school year. The focus of the evaluation was on the study of 
the implementation of the program, its effectiveness, and its 
suitability for replication in schools throughout the state. This 
report begins by reviewing the background and purpose of the study 
*nd describing the program, including its goals and components. The 
'^hree principal components are identified as training of trainers, 
inservice training, and implementation. The evaluation design and 
procedures are then presented. The evaluation focused on questions 
related to the three components, and the evaluation instruments used 
are described in detail by program area. The results and conclusions 
of the evaluation are presented for each component. On the basis of 
the evaluation data collected, recommendations are made in the areas 
of computer availability, computer literacy, program coordination, 
and focus of training. The evaluation forms used in the study are 
appended. (JLB) 
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BACKGROUND AND PURPOSE 



This document represents the final report for an evaluation study of 
the Delaware Computer Inservice Program, designed to increase the use of 
computers in traditional subject areas. The program was pilot tested in 
three schools, each from a different district in Delaware during the 1987-88 
school year. The three districts involved were: Capital, Red Clay,. and 
Seaford. The evaluation was jointly undertaken by the Delaware Department 
of Public Instruction (DPI) and Research for Better Schools (R3S) . RBS was 
to assume responsibility for design and analysis of evaluation instruments, 
while the DPI qualified schools for participation, monitored training and 
implementation, and administered and collected evaluation instruments. The 
focus of the evaluation was on the study of the implementation of the pro- 
gram, its effectiveness, and its suitability for replication in schools 
throughout the state. 

As with any evaluation study, the validity of assumptions underlying 
implementation of the program studied as well as the actual quality and 
extent of implementation provide an important context for interpretation of 
evaluation findings. In the case of the present study, it should be noted 
that the program was fully implemented as planned in only one of the three 
participating school sites, although the other two sites completed the 
training required for implementation. 

The balance of this report provide s the background for the program and 
details the evaluation design and procedures. It also describes the evalua- 
tion findings and presents conclusions and recommendations drawn from these 
findings . 
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PROGRAM DESCRIPTION 

The Delaware Computer Inservice Program had its beginnings in the 
spring of 1984. At that time, the Delaware Department of Public Instruction 
and Project DIRECT, with the assistance of Research for Better Schools, 
initiated a project to increase the use of computers in traditional subject 
areas. The project took the form of a two-year effort focusing on the 
development of two resource guides for instruction. One of these resource 
guides was for social studies, and one was for language arts. Each of the 
guides contained: an overview of how computers could be used to support 
instruction in the content area, an approach to developing instructional 
plans using computers, a framework for previewing software, an overview of 
management considerations to be addressed in computer use, and examples of 
potential use with software and resource ■ groups . 

In the fall of 1986, a model inservice program was designed for schools 
based on the resource guides. This program was initiated by the Division of 
Instruction and the Division of Computing Services (formally Project DIRECT) 
at the DPI in order to help school staff successfully implement the sugges- 
tions found in the guides. The model was to be pilot tested in three schools 
during the 1987-88 school year with the intention, if successful, to replicate 
it on a statewide basis. This report constitutes the evaluation of that pilot 
tes t . 

Program Goals 

In the conceptualization and design of the Delaware Computer Inservije 
Program, its originators were guided by five goals. These goals were as 
follows : 
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• to increase the use of computers for instruction by social studies 
and language arts teachers 

• to increase the variety of available computer software used by 
social studies and language arts teachers 

• to develop local training and support systems for facilitating 
increased computer use in social studies and language arts 

• to develop a model, a process, and materials (including those 
matching software content to state standards) to facilitate 
increased use of computers in social studies and language arts 

• * field test the model, process, and materials for possible 
s' Lv ->wide implementation. 

In associa. Jon with the achievement of these program goals, certain 
outcomes were anticipated involving positive impacts upon teacher behavior, 
student behavior, and other areas within the school setting. The above 
goals and their associated outcomes were incorporated into a set of eval- 
uation questions which provided the focus for the evaluation study. 

Program Components 

The program had three principal components: training of trainers, inser- 
vics training, and implementation. The training of trainers was to be accom- 
plished through a workshop conducted by state and RBS staff. Its purpose was 
to prepare a cadre of local school personnel who would serve as the inservice 
leaders. Next, these leaders were to provide inservice training to teachers 
in their own districts, in order to help them understand and apply the content 
of the guides. Finally, the leaders were to serve as local support persons, 
helping teachers to implement instructional plans that incorporated the use of 
computer technology. All three components of the program needed to be carried 
out effectively if the desired outcomes were to be realized. 



Training of Trainers 

A three-phase training workshop was to be given to school personnel 
selected to serve as trainers or inservice leaders for eventual training of 
teachers' in use of the program. A number of objectives were to be accom- 
plished through the training of trainers sessions. 

First, they were expected to learn how to orient teachers to the dif- 
ferent ways the computer can be used in their specific subject areas, how to 
explore different types of software programs, and how that software might be 
used in classroom lessons. In the first phase of training, they were expected 
to learn to describe the program's goals and objectives, help the district and 
school administrators prepare remarks for introducing the inservice program to 
teachers, and demonstrate major types of software programs. In the second 
phase, they were to learn to demonstrate the process of previewing software 
and of instructional planning. They were to learn to provide an overview of 
an approach to selection of software and to the development of instructional 
plans based on the Delaware Computer Resource Books. They were also to demon- 
strate the Delaware State Standards and Software Matching System. This system 
was designed to provide an indexing of computer software to the state stan- 
dards for graduation and academic progress. They were to learn to describe 
how to use resources from the Educational Products Information Exchange (EPIE) 
in helping to accomplish these tasks and to learn to provide assistance and 
support in guided practice on tasks related to software previewing and 
instructional planning. In the third phase of the training, the trainers were 
to learn to help teachers develop . i implement an instructional plan. They 
were to provide assistance and support to teachers in the selection of soft- 
ware and development of instructional plans consistent with the approach 
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demonstrated in the previous phase. They were to make arrangements for obser- 
vation of teachers implementing their instructional plans, and provide feed- 
back to teachers that would help them in revising their plans. They were also 
to facilitate teachers sharing ideas and experiences based on implementation 
of their instructional plans. 
Inservice Training 

The inservice training sessions were designed to fully train teachers to 
utilize the model program, to utilize computer hardware and software in 
their instruction, and to rely on available resources to facilitate this 
usage. Teacher inservice sessions were conducted over an extended period of 
time and involved not only workshop sessions but practicum experiences. 

Specific objectives of the teacher training, in terms of what the 
teachers were expected to be able to do after the training, included the 
following : 

• articulate the school's goals and objectives for the program 

• understand the use of the computer and computer software in 
instruction 

• understand and use the proper procedures for selection of software 

• understand the process for developing an instructional plan that 
incorporates computer activity and use 

• develop an instructional plan that matches specific curriculum 
obj ectives and incorporates computer use 

• .implement an instructional plan in the classroom 

• critique the experience 

• revise and implement instructional plans 

• develop increased understanding of the range of computer uses for 
instruction 

• identify ways of improving instructional plans- 
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Implementation 

Implementation of the program was to consist of the trained teachers in 
each of the three pilot schools assuming responsibility for: developing 
lesson plans which were to incorporate the use of computers, implementing 
those lesson plans, and reflecting upon their implementation experiences to 
modify the plans for future use. In order for these events to take place, 
it was essential for the necessary administrative support and resources to 
be made available to the teachers as well as follow-up technical assistance. 
Teachers needed access to the school's computers during their class ses- 
sions. They needed access to software relevant to their class lessons. 
They also needed administrative support from their principals and technical 
assistance from their trainers. 



EVALUATION DESIGN AND PROCEDURES 



The evaluation was designed to address both process and outcome aspects 
of the program, although incomplete implementation of the program served to 
make outcome evaluation efforts almost meaningless. 

The evaluation was to focus on questions related to the three major 
program activities: training of trainers, inservice training, and implemen- 
tation. It was to document the activities undertaken in connection with the 
program and to assess the quality and effectiveness of those activities. 
Feedback from the evaluation was expected to be useful to program adminis- 
trators and staff in adjusting and fine tuning the program activities to 
make them more effective and for later use in future implementation. 

Three schools were to be involved in the pilot test and hence the eval- 
uation: one school each from the Capital, Red Clay, and Seaford districts. 
Each school site was to select trainers with certain qualifications and send 
them to the training of trainers workshop. Each was also to similarly iden- 
tify teachers to be trained later by that trainer and to make available 
necessary resources at the local site to be able to implement the program as 
planned. DPI staff were to assume responsibility for site selection, com- 
patibility, and for monitoring program implementation. 

Evaluation Questions 

Based on the goals of the Delaware Computer Inservice Program, and the 

outcomes anticipated from their achievement, seven evaluation questions were 

developed to serve as *:he focus for the evaluation study of the program. 

These questions were as follows. 

• To what extent did the training of trainers process adequately 
prepare the trainers for their training and support role7 
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• To vhat extent did the inservice training sessions adequately 
prepare the teachers for implementing the program? 

• To vhat extent was the program implemented as planned with the 
necessary time and resources available? 

• To what extent did the program cause teachers to increase use of 
computer hardware and software in their instruction? 

• To what extent did the program provide for an increase in student 
exposure to computer-based instruction? 

• To what extent was the computer inservice model effective in accom- 
plishing its goals and objectives? 



Instruments and Procedures 

Evaluation instruments and procedures were developed to be both com- 
patible with intended program activities and relevant to addressing the 
above evaluation questions. Table 1 shows each of the three principal 
program activities, the evaluation questions associated with each, and the 
data sources used to address the questions. A more detailed description of 
the instruments and procedures used to evaluate each program activity is 
provided in the paragraphs that follow. Samples of the instruments are 
contained in the Appendix. 
Training of Trainers 

For evaluating the training of trainers activity, the following instru- 
ments and data sources were utilized. 

• Training Feedback Form (II) 

• Materials Feedback Form (II) 

• Observations 

• Trainer Interviews 

In order to assess the quality and effectiveness of the training of 
trainers workshop, a training feedback form and materials feedback form were 
developed and administered to participants. These were distributed to 
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Table 1 

Program Activities, Questions and Data Sources 



Program Activity 

Training of 
Trainers 



Inservice 
Training 



Evaluation Question 

To what extent did the training of 
trainers process adequately prepare 
the trainers for their training and 
support role7 



To what extent did the inservice 
training sessions adequately prepare 
the teachers for implementing the 
program? 



Data Sources * 

Training Feed- 
back Form 
(post) (4) 

Materials Feed- 
back Form (3) 

Observation? 

Trainer 

Interviews 

Teacher Inser- 
vice Evalua- 
tion Form 
(mid) (19) 

Teacher Inser- 
vice Evalua- 
tion Form 
(post) (9) 

Trainer 
Interviews 



Implementation 



To what extent was the program im- 
plemented as planned with the 
necessary time and resources avail- 
able? 

To what extent did the program cause 
teachers to increase use of computer 
hardware and software in their in- 
struction? 

To what extent did the program pro- 
vide for an increase in student ex- 
posure to computer-based instruction? 

To what extent was the computer in- 
service model effective in accom- 
plishing its goals and objectives? 



Teacher Inser- 
service Eval- 
uation Form 
(post) (9) 

Teacher Log 
(A3) 



Teacher Lesson 
Plans (39) 



Trainer Inter- 
views (3) 



* Number in parentheses following each data source indicates total number of 
returned forms. 
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participants at the end of the third phase of the training of trainers work- 
shop series and were designed for assessing the benefits of the workshop for 
performing their training roles. 

The materials feedback form contained 21 items on adequacy of the 
materials provided in terms of supporting the trainer's role as trainer and 
technical assistance specialist. The items had a four-point Likert-type 
response format with the scale ranging from "not adequate" to "very adeq- 
uate . n 

The training feedback form contained 17 items which were to be 
responded to on a four-point Likert scale. The items related to the various 
objectives of the training of trainers workshop series. Each rating scale 
represented a range from "not successful" to "very successful." 

In addition, the questionnaire information represented by the above two 
forms was supplemented with limited observations of the training sessions by 
RBS staff assisting in the training. Also, one trainer from each site was 
interviewed at the conclusion of the pilot program year and asked to evalu- 
ate the training of trainers workshop series. 
In service Training 

The inservice training of teachers was evaluated using the following 
instruments and data sources: 

• Teacher Inservice Evaluation Form (II) 

• Teacher Inservice Evaluation Form (III) 

• Trainer Interviews 

In order to effectively evaluate the quality and effectiveness of the 
teacher inservice training, two teacher questionnaires were constructed and 
administered. One questionnaire was given mid-way through the inservice 
series, just prior to the practicum experience, The second was given at the 
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end of the program year. The first questionnaire contained 18 Likert-type 
items, while the second year-end questionnaire contained 11 Likert-type 
items. Comments and an open-ended question regarding suggestions for 
improvement were also provided for in this latter questionnaire. 

Additionally, one trainer from each site was interviewed at year-end 
and asked questions evaluating the teacher inservice training he or she 
conducted. 
Implementation 

The instruments and data sources relevant to implementation of the pro- 
gram were as follows: 

• Teacher Inservice Evaluation Form (III) 

• Teacher Log 

• Teacher Lesson Plans 

• Trainer Interviews 

Data on implementation were collected through the above end-of-year 
questionnaires as well as through telephone interviews with trainers. In 
addition, teacher logs and lesson plans were collected and reviewed for 
implementation evaluation purposes. These latter data sources were examined 
for evidence of increased frequency or intensity of computer usage by par- 
ticipating teachers. Interviews with trainers addressed, among other 
things, changes in the level of teacher comfort with use of computers ♦ 
changes in attitude toward computer use, and changes in level of knowledge 
about available software and how to preview it. 
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RESULTS AND CONCLUSIONS 

This section presents the results from the compilation and analysis of 
the data collected as part of the evaluation of the Delaware Computer 
Inservice Program. It also presents conclusions drawn from those results to 
address each of the initial seven evaluation questions posed by the study. 

The numbers of evaluation forms returned for analysis are indicated in 
parentheses under the data sources column of Table 1. The returns were well 
below expectations. This was primarily due to the fact that, although all 
three sites participated in training of trainers and inservice training, the 
program itself was fully implemented at only one of the sites. Because of 
the low number of respondents, it was not feasible to analyze the data by 
individual sites. Instead, data were aggregated across sites for analysis 
purposes. Although there were inter-site differences in level of implemen- 
tation of the program, the data collection instruments were specific enough 
to be able to match the level of implementation achieved. This attribute as 
well as the number of non-respondents to certain of the questionnaires 
insured that respondents were only returning forms appropriate to their 
level of program implementation. 

Analyses of the data were conducted and yielded descriptive statistics 
for all items on all questionnaires. These are presented in the Appendix, 
on each evaluation form, in the form of response frequencies and mean 
responses to each item. 

Training of Trainers 
Eight school staff in all were selected to be trained as trainers. 
Three each were from Capital and Seaford school districts and two were from 
Red Clay school district. One Seaford trainer dropped out after taking 
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training because he had little or no background in computers. Of the 8 
persons beginning training as trainers for the program, only four returned 
training feedback forms and only three returned materials feedback forms. 
Frequencies and item statistics for those respondents returning training 
feedback forms and materials feedback forms are presented in the Appendix. 

A summary of the training feedback responses in terms of whether or not 
the training was successful in accomplishing the workshop objectives is 
presented in Table 2. This table displays the workshop objectives, organ- 
ized by the categories of: orientation, preview and planning, and develop- 
ing and implementing plan. Accompanying each are judgments about its accom- 
plishment based on the data analysis. As can be seen, all but four of the 
objectives measured in the instrument were achieved by the training of 
trainers activity. Two of these items, which were not successfully accom- 
plished, dealt with the state matching system and the use of EPIE resources. 
The state matching system was not fully developed at the time of training 
and was therefore never fully utilized by those who were trained at two of 
the three sites, while the EPIE resources were received too late in most 
cases to be used by the trainers in their own training. The other two 
objectives which were unsuccessfully obtained had to do with observing 
teachers implementing their instructional plans and providing feedback on 
those observations. Several of the trainers believed that this was not part 
of their responsibilities as trainers. Although originally planned as part 
of the program, they believed that this was changed because of their course- 
loads and the perceived inappropriateness of their supervising their col- 
leagues in this manner. In all, 13 out of 17 of the training objectives 
assessed by the training feedback form were successfully attained by the 
training of trainers workshop series according to the respondents. 



Table 2 



Success of Trainers in Conducting Teacher Inservice Workshops 

Successful? 

A. Orientation 

1. Introducing program goals and objectives YES 

2. Helping administrators prepare inservice 

opening remarks YES 

3- Discussing participant experiences with computers YES 

4. Demonstrating software program type YES ■ 

5. Discussing uses of computers in subject areas YES 

B. Previewing and Planning 

1. Introducing approach to software selection YES 

2. Demonstrating Delaware Matching System NO 

3. Describing contents and use of EPIE resources NO 
4* Guiding software selection YES 

5. Guiding development of instructional plan YES 

6. Describing preparation for implementation YES 

C. Developing and Implementing Plan 

1. Providing assistance and support YES 

2. Reviewing and critiquing plans YES 

3. Observing teacher implementation plans ^ NO 

4. Providing feedback on observation NO 

5. Helping teachers share and learn from their 
experiences YES 

6. Describing schedule of work in implementation YES 
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The materials feedback form was returned by only three trainers. 
Results from the materials feedback form show 20 of 21 items rated as 
adequate in terms of inservice materials. The sole item rated not adequate 
was the planned' demonstration of the Delaware Star.dards/Sof tware Matching 
System. At the time of training, this system was not completed and was not 
available for use by most of the teachers as discussed earlier. The un- 
availability of this matching system was a disappointment to a number of the 
participants . 

Informal observations of training by RBS staff as well as end-of-year 
interviews with one trainer from each participating district indicated that 
the training was quite adequate for the group as a whole. However, the 
training should be revised to provide greater depth. This was viewed as 
essential for those participants with little or no knowledge of computers. 

Because of the apparent diversity of computer literacy levels evidenced 
by the prospective trainers at the outset of training, the training was 
adapted to meet the group needs, with more time spent on the basics of 
computer use than originally planned, With these changes, the training of 
trainers process was judged by the trainers to have adequately prepared them 
for their training and support roles as reflected in their responses to the 
training questionnaires. In addition, informal observations of the training 
made by RBS staff concur with these findings. 

Inservice Training 
Inservice training of teachers was conducted at each of the sites by 
the trainers participating in the training of trainers workshop. In all, at 
the three sites, a total of 30 teachers participated in inservice training. 
Fifteen teachers participated from Red Clay school district, 9 from Capital, 
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and 7 from Seaford. Teacher inservice evaluation forms were administered to 
participating teachers in the middle of the training program as well as at 
the end of training. A total of 19 respondents returned the mid-teacher 
inservice evaluation form, and only nine returned the post inservice evalua- 
tion form. Frequency distributions and individual item means are presented 
for each of these instruments in the Appendix. 

A summary of the mid and year-end teacher inservice evaluation forms is 
presented in Table 3. This summary indicates for each of the training 
objectives whether or not that objective was successfully achieved based on 
respondents' answers to relevant items on the questionnaires. As can be 
seen in this table, all ten of the training objectives were indicated by the 
respondents to have been successfully attained. However, feedback from the 
participating teachers indicated the desire for more directed training on a 
more limited and selective array of software products and more modeling of 
instructional plans incorporating use of computers. 

Year-end telephone interviews with selected trainers on the adequacy of 
inservice training indicated mixed feedback. The consensus among those 
trainers who implemented the program was that the training was indeed ade- 
quate. However, for those teachers with little or no previous experience 
with computers, the trainers felt that the training they provided was too 
short. The trainers also felt that for those teachers with little or no 
access to computers in their schools, the training was of little use. The 
training was designed with several assumptions in mind (including partici- 
pant computer literacy and computer availability). Where these assumptions 
were met, the training proved more than adequate. 
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Table 3 

Inservice Training Activity Success 



Training Objective 

Describe school goals and objectives for the program 

Describe use of computer and software in instruction 

Describe process for software selection 

Describe process for developing instructional plan 
incorporating computer use 

Develop instructional plan matching curriculum 
objective and computer activity 

Implement plan in classroom 

Critique experience 

Critique instructional plans 

Develop understanding of computer use 

Identify ways of improving instructional plans 



Successful? 
YES 
YES 
YES 

YES 

YES 
YES 
YES 
YES 
YES 
YES 
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Implementation 

Documentation of the implementation of the program was accomplished 
through the collection of teacher lesson plans, teacher logs, year-end 
teacher inservice evaluation forms, as well as limited observations and 
telephone interviews with trainers. Of the 30 teachers involved in the 
program, 10 completed and submitted instructional plans and 13 submitted 
teacher logs. End-of-year telephone interviews were conducted with one 
trainer from each district. 

Based on this data, it is apparent that the sites attained very dif- 
ferent levels of implementation of the program. At one site the program was 
implemented as planned, but with additional time required to review software 
and apply it for use with students. At a second site, the program received 
very limited implementation, with the number of participants completing the 
program dropping from twelve to five because they could not implement their 
lesson plans due to time constraints and limited access to computers. At 
the third site, the program was not implemented because of problems with 
software availability in the computer lab,' and no access by these teachers 
to computers in the lab. Thus, the program was fully implemented at only 
one of the three sites. 

Despite the problems with implementation, based on the teacher logs, 
over 400 students experienced instruction through use of computers inte- 
grated into their classroom activities. These students were from grades 1 
through 6 and 9 through 12. From examination of the lesson plans and 
teacher logs returned to DPI, it is clear that the program was implemented 
generally as planned for those teachers who were provided access to the 
necessary computers and software. It is also clear from this same data that 

the program caused teachers to increase use of computer hardware and 
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software in instruction over the level of use prior to the program. As a 
result, the program provided for an increase in student exposure to 
computer-based instruction. Responses by teachers to open-ended items in- 
cluded on the teacher log indicated that the program plan was implemented as 
written, that plan objectives were achieved, that the plan promoted student 
achievement, that the plan promoted student motivation, and that the plan 
proved more effective than regular instruction. This latter effect may have 
been due to its novelty and change of pace from traditional instruction. 
End-of-year telephone interviews with trainers generally indicated addi- 
tional support for the above contentions. 

With respect to the extent to which the program caused teachers to 
increase use of computer hardware and software in their instruction, the 
responses once again varied depending on availability of resources at the 
pilot- site schools. At the first site usage of hardware and software 
increased, at the second site it did not because of the limited access to 
computers, and the third site reported little increase in use except for the 
computer lab teacher. Similar responses were obtained with regard to the 
question of increased student exposure to computer-based instruction and the 
enhancement of other areas of computer based instruction, as might be 
expected . 

The extent to which the model was effective in accomplishing its goals 
and objectives was difficult to judge. Evaluation data indicate that both 
the training and implementation aspects of the model were effective in the 
one site which met all of the assumptions upon which the model was predi- 
cated. At the other two sites , responses to training evaluation forms 
indicated that the training processes were generally effective for all sites 
(though modifications are warranted). However, implementation at these 

O Op 

ERLC 



other two sites was limited due to insufficient levels of computer literacy, 
and little or no access to computers at the sites. Consequently, there was 
little increase in the use of computer-based instruction at these two sites. 
It should be noted, however, that the program is operating with software 
resources now at one of these two sites, now that the computer lab has been 
made more accessible to teachers. 

Those trainers who were interviewed at all three sites felt that the 
program was needed and had potential, if the problems with preparation and 
access to computers could be remedied. 
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RECOMMENDATIONS 



From its inception, the Delaware Computer Inservice Program was fraught 
with problems involving both training and implementation aspects of the pro- 
gram. The occurrence of these problems severely limited any assessment of 
outcomes for the program and led to serious questions regarding some of the 
assumptions made at the outset about the kind of training, site preparation, 
and monitoring necessary. 

To the extent the program was implemented as planned, it apparently was 
favorably viewed by both teachers and students as evidenced by the data 
collected in this evaluation. It is therefore likely that this kind of a 
computer inservice program will be piloted again. On the basis of the 
process evaluation data collected as part of the present evaluation, several 
strong recommendations can be offered for future efforts of a similar nature 
in incorporating the use of computers in classroom instruction. These are 
presented and discussed below. 

Computer Availability 
A number of teachers commented that their access to computers for a 
try-out of their lesson plans was very limited. Some could not complete the 
five lesson plans agreed to because of this limited access. Some had no 
access at all. If computers are to be integrated into the instructional 
planning, much greater access must be provided. Therefore, the following 
recommendations are offered . 

• Access to computers for more than one period per week be made a 
minimum prerequisite exposure level for any future program 
implementations involving the integration of computer use into 
instructional subject areas. 

• During access periods, sufficient computers be provided such that 
there is at least one computer for every two students in the class. 

o .? 4 
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Computer Literacy 

For a number of teachers participating in the inservice training and in 
some cases the training of trainers workshop sessions, their prior famili- 
arity and experience wifh computers was limited. During training, they not 
only had to absorb the contents of the training sessions but also become 
familiar with the computer hardware and software at the same time in order 
to get full benefit from the training. Their limited access to computers at 
their home schools further exacerbated the problem. The following recommen- 
dations are therefore offered: 

• Participating teachers first receive inservicing on computers in 
general, emphasizing computer literacy. 

• Program inservice sessions spend more time on familiarization with 
the computer hardware initially. 

Program Coordination 
Management and coordination of the program implementation at each of 
the sites was handled differently. The site which employed a local coordi- 
nator for the program seemed to work best in terms of general implementa- 
tion, meeting timelines, sharing of information, and responsiveness to 
evaluation and other requests. Such a person can often serve as a local 
advocate for a new program and promote the kind of enthusiasm and task- 
directed orientation to keep implementation moving forward. The following 
recommendation is therefore offered: 

• Each participating program site should be required to provide a 
local coordinator with appropriate release time to assist in 
coordinating implementation of the program and calling for state 
assistance and support where needed. 
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Focus of Training 

The focus of the training programs for both trainers and teachers was 
on providing an awareness of software available for use, on how to preview 
and select the software, and on how to incorporate it into lesson planning. 
The assumption was that participants already had a familiarity with com- 
puters, had workshop time available to preview software, and had access to 
computers for previewing and learning software. These assumptions were 
unfounded. In many cases, both trainers and teachers did not- have the time 
or equipment to preview the software and learn it to a level at which they 
could comfortably use it in instruction. Moreover, trainers did not have 
time to observe and critique teachers in their implementation of the program 
nor did they feel comfortable in doing so. Therefore, the following 
recommendations are offered. 

• . The training workshops for both trainers and teachers should be 

longer and should devote more time to learning, in-depth, several of 
the basic software programs capable of being used in the subject 
areas . 

• The training of trainers workshops should de-emphasize the formal 
observation and critique of teachers in their implementation of the 
program and replace this emphasis with support and technical assis- 
tance activities . 
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CLOSING REMARKS 



Despite its implementation problems at the school sites, the Delaware 
Computer Inservice program represented a step forward in integrating com- 
puter use into the social studies and language arts classrooms at all levels 
within the education system. As with any new initiative, planning, prepara- 
tion and resources are critical to its implementation. For a model program 
to be effective, the assumptions upon which the model was built must be met. 
In this regard , the three sites implementing the computer inse rvice program 
were in different states of readiness to receive the program. The quality 
and effectiveness of implementation varied considerably. However, despite 
the obstacles encountered, the program met with a favorable -reception by 
those staff and students who were able to gain access to computers and use 
them in their lessons. 

The recommendations from the current evaluation study should help 
program administrators in making the necessary changes to the program to 
provide for its full implementation and to allow for a more meaningful 
assessment of its potential for promoting a better education for students 
throughout Delaware. 



ERLC 



^7 

* » 1 



APPENDIX 



RESULTS (N-19) 
Social Studies . School: 



English/Language Arts District: 



Date: 



TEACHER INSERVICE EVALUATION FORM (II) 

DPI would like some feedback from teachers participating in the Computer 
Inservice Program. This questionnaire addresses your experience with the 
first three phases of the program: orientation, demonstration of instruc- 
tional planning, and development and implementation of an instructional plan 
with quidance. Please circle the appropriate response for each item and note 
any comments, where requested. 

Thank you for your cooperation. The information that you provide will help 
DPI plan follow-up activities and improve future training. 



Directions: Rate the extent to which the following objectives of the 
training program were achieved. 

Not 

I can now: Completely % At All 



describe the district/school's plans 4 3 2. 1 

regarding the use of computers and 16 68 16 3.00 

their expectations for the inservice 
program for the school 

describe the uses of computers and 4 3 2 1 

computer software for instruction in 42 53 5 3.53 

my subject area 

describe a process for selecting 4 3 2 1 

software for instruction using the 3.16 



Delaware State 'Standards Hatching 
System, Delaware preview form, and 
software reviews 



32 53 16 



describe a process for developing 4 3 2 1 

an instructional plan that includes ^ 32 5 3 - 7 ^ 

use of computer software 

implement an instructional planning 4 3 2 1 

process (identify, preview and select 53 21 11 3,58 

software; develop a lesson plan to 

meet criteria; successfully implement 

the plan) 
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B Rate the value of each of the following major activities of the inservice 
program vith respect to your achieving the program's objectives. Wherever 
possible, provide explanatory comments. 



Very 
Valuable 



Not 
At All 
Valuable 



Did Not 
Occur 



x 



I found the : 

1. orientation, including discussion 
of experiences vith software and 
demonstration of alternative uses 
of compilers in a subject area 



4 
47 



3 
47 



2 
5 



DNO 



3.58 



Comments : 



2. overview of developing an 

instructional plan, including 
demonstration of each of the 
planning steps 

Comments : 



4 3 2 1 DNO 

42 37 . 5 16 3.44 



4 3 2 1 DNO 



3. development of an instructional 
plan that incorporates the use of 
computers and receiving feedback 
from my colleagues 

Comments : 
Not yet heard from colleagues- 

4. implementation of my instructional 
plans and receiving feedback from ^ 5j > 0 22 3.42 
an observer 

Comments : 
Saw none; Haven f t worked yet in lab; 
Not been observed yet; Had no feed- 
back for first three lessons. 

5. sharing of the implementation 
experiences vith my colleagues 

Comments : 

Haven't done this yet; Done informally. 



4 3 2 1 DNO 

33 50 6 11 3.31 
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C Rate the value- of each of the major sections of the Delware Computer 
Resource Book vlth respect to your achieving the p rogram's objectives. 
Vherever possible, provide explanation. 



I found : 

1. Section I. Using a Computer to 
Teach Social Studies (or Language 
Arts) 

Comments: 



Very 
Valuable 

4 
16 



% 



3 
68 



2 
16 



Not 
At All 
Valuable 

1 . 



Did Not _ 
Read x 



DNR 



3.00 



2. Section II. Designing^ 
Instructional Plans Which 
Incorporate Computer Programs 

Comments : 

Not enough computers. 

3. Section III: Identifying and 
Previewing Computer Programs 

Comments : 



4 
42 



4 
57 



3 
42 



3 
32 



2 
16 



2 
11 



DNR 



3.26 



DNR 



3.47 



4. Section IV: Management 
Considerations 



4 
16 



3 
68 



2 
11 



DNR 
5 



3.06 



Comments : 

5. The Appendices 

• Bibliography 

Comments : 
Didn't use. 

• Descriptions of Centers/Offices 

Comments : 
Didn't use. 



4 
29 



4 
6 



3 
47 



3 
59 



2 
12 



DNR 

24 3.38 



DNR 

18 2.79 
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Preview Form 



Comments : 



Very 
Valuable 

4 

39 



3 
61 



Not 
At All 
Valuable 



Did Not _ 
Read _2L 



DNR 



3.39 



• Software to Explore 
Comments : 



4 
56 



3 
33 



2 
6 



DNR 

6 3.53 



General Comments 

More credit should be awarded for this course — I have invented far more than 
15 hours. 



*3 0 
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RESULTS (N=9) 



Social Studies 
English/Language Arts 



School: 

District: 

Date: 



TEACHER INSERVICE EVALUATION FORM (III) 



Earlier in the year you completed a questionnaire vhich focused on the first 
three phases of the Delaware Computer Inservice program. Nov that you have 
completed the program, DPI would like to obtain some feedback concerning your 
experiences with phases IV and V: independently developing and implementing 
instructional plans using computers and computer software, and the sharing of 
these instructional plans. 

Thank you for your cooperation. Your opinions are very much needed. If you 
vould like to include suggestions for improvement, please do so in the last 
section of the questionnaire. 



A. Directions: Rate the extent to vhich you feel you can implement each of 



the folloving steps in the instructional planning process. 

Completely 



I can now, without assistance: 



% 



Not 
At All 



identify, preview, and select soft- 
vare to support instruction 

develop a lesson plan vhich integrates 
computer activities and meets the 
criteria 

successfully implement the lesson 
plan 



4 


3 


2 


1 






67 


33 






3 


67 


4 


3 


2 


1 






67 


22 


11 




3 


56 


4 


3 


2 


1 






67 


22 


11 




3 


.56 
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« p.r. the value of each of the following major activities of the inservice 
I/jTlel iec. to v"" achieving th. programs oblectlves. Wherever 
possible, provide explanatory comments. 



Not 

Very • At All Did Not 

Valuable %_ Valuable Occur 



I found the : 

1. development of instructional plans 4 3 2 1 DNO ^ ^ 

and trying them out vith my classes °' 

Comments : 

This gave me time to preview the disks that were available to me. 
Therefore, I was able to use more computer time incorporated into 
my lessons. 



2. sharing of the instructional plans 



4 3 2 1 DNO 



and implementation experiences vith 67 33 
my colleagues 

Comments : 

By sharing comments about a particular disk I was able to pick and 
choose the ones that would be relevant to my situation. . 

C Rate hov accessible each of the following were vith respect to your independently 
developing and implementing instructional plans. 



3.67 



Very 



Not 



Accessible % Accessible 

I found the : 

1. computer software which I needed for ^ J 2 1 ^ 

developing instructional plans 

Comments : 

Sometimes others checked it out before I got 
there. 



2. scheduling of five (or more) class * 3 2 ^ 

periods with computers (lab) °' 1 

Comments : 

Had never used a computer before; Had one 
computer in my room all year. 
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I found the : 

3. computer disks and other related 
materials 

Comments : 



Very 
Accessible 



4 
44 



Not 
Accessible 



.3 

44 



2 
11 



3.33 



D. Rate the extent to which you would: 



1. use next year the instructional 
plans that you developed 

Comments 



Defini tely 

3 
78 



Probably 

2 
11 



Unsure 



11 



2.6 



2. use next year some of the 

instructional plans that your 
colleagues developed 

Comments : 



2 
78 



1 

22 



1.7 



develop new plans that integrate 
computer activities for use next 
year 

Comments : 



3 
78 



2 
22 



9 
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What suggestions do you have for improving this inservice program? 
(continue comments on back of page) 
More instruction on the printer; 

Time it took teachers to review software was too time-consuming and prevented 

development of activities; 
Have more printers available (2) ; 

Allow us to preview disks for subjects we were interested in earlier (2); 

(cont'd) 
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E. What suggestions do you have for improving this inservice program? (cont'd) 



Obtain more computers (4) ; 

More guidance in developing lesson plans that integrate computer use; 

Include lessons on word processing during training (2); 

Make disks more accessible for immediate use; 

More practice with management system; 

Make hardware more accessible; 

Provide extended memory for computers; 

More practice; 

Need more teachers involved in training. 



Numbers in parentheses indicate number of respondents making that 
comment . 
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RESULTS (N=4) 
.Social Studies School . ^ 

.English/Language Arts District; 

Date: 



TRAINING FEEDBACK FORM Cm 



A. PHASE I: ORIENTATION 

How successful vere you in: 



. Very P , Not 
■ Success£ul _%_ Successful x 



Presenting the inservice program's 

goal, the logic of the five phases 4 3 2 j 

(and of the role that the leader(s) 75 25 3.75 

will play), the objectives of the 

phases, and the timeline for the 

phases 

Comments: 



Helping the principal/central office a 

administrator prepare his/her remarks 75 J 1 

for the beginning of the inservice 25 ^5t 



program 
Comments: 



• Leading a discussion which draws out 
the participants' experiences using 
computers in their classrooms and 
their evaluation of those uses 

Comments : 

Few participants had experience to rel 



3? 



4. Demonstrating the major types of soft- 
ware programs and eliciting from the 
participants: 

e the potential learning outcomes 
possible from each program 

• the prerequisites, support, and 
follov-up required to achieve 
those outcomes 

• the advantages and disadvantages 
of the software in comparison to 
traditional classroom practices 

Comments: 



Very 
Successful 

4 

50 



Not 
Successful 



3 

50 



3-5 



Leading a discussion summarizing 
primary uses of computers in a 
subject area, the advantages as- 
sociated with such uses, and the 
conditions required to maximize 
the advantages and minimize the 
disadvantages 

Comments : 



4 

25 



3 

75 



3-.2f 



B. PHASE II: DEMONSTRATION OF PREVIEWING 
AND INSTRUCTIONAL PLANNING 

How successful were you in : 

1. Providing an overview of the approach 4 
to selecting software and developing 
instructional plans presented in the 
Delaware Computer Resource Book 

Comments : 

Delaware Selection System was incomplete and 
not in place; Had to use own selection process, 



3 

.100 



3.0 
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Very 
Succes sful 

7 



2. Demonstrating the Delaware State Stan- 
dards Software Hatching System 

Comments : 

System Incomplete, Unavailable, Non-existant . 



2 

67 



Not 
Successful 

1 

33 



1.67 



3 Describing contents and use of EPIE 
Micro-Courseware PRO/File and Evalua- 
tion and TESS — The Educational Soft- 
ware Selector 

Comments : 

Got my copy too late to use. 



3 

33 



2 

33 



1 

33 



2.00 



4. Guiding a group through the task of 
previewing a selected piece of soft- 
ware 

Comments: 



4 
50 



■ 3 
50 



3.50 



5. Guiding a group through the task of 
developing an instructional plan 
that incorporates the use of the 
software previewed and reflects the 
approach presented in the Delaware . 
Computer Resource Book 

Comments : 



4 
25 



3 

75 



3.25 



6. Describing the preparatory task for 
Phase III, developing and implement- 
ing an instructional plan, with gui- 
dance 



25 



3 

75 



3.2 



Comments: 
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Very 



Not 



Successful _JL Successful 



4 3 2 1 

50 25 25 ' 3*25 



C. PHASE III: DEVELOP AND IMPLEMENT AN 
INSTRUCTIONAL PLAN VITH GUIDANCE 

How successful were you in : 

1. Providing the support teachers need 
to select software and develop an 
instructional plan consistent with 
the approach demonstrated in Phase II 

Comments : 

Energy was high at the beginning and later 
waned. 



2. Reviewing and constructively critiquing 6 7 oo 3,67 

draft instructional plans and/or b/ 33 — 

facilitating a group review and 
critique of draft plans 

Comments : 

Hard to criticize work of other teachers. 



3. Making arrangements 'for observations 4 3 2 1 

of teacher implementation of instruc- 1UU i_^uu 

tional plans and/or observing implemen- 
tation in order to gather information 
useful for revision of plans 

Comments : 

Thought we were relieved of this respon- 
sibility; Impossible due to class load; 
More comfortable doing exercises on own 
and sharing results. 

4. Providing feedback to teachers observed -4 3 2 1 

that helped them make revisions to their 100 1-0C 

plans 

Comments : 

No observation/no feedback. 
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Very 
Successful 



Not 
Successful 



5. Helping teachers share and draw lessons " A 3 
from their implementation experiences 33 67 



3.33 



Comments : 



6. Describing the schedule of work, for 4 3 2 1 

Phase IV, that each teacher or teacher 100 . 3.00 

team needed to develop 

Comments : 



General Comments 



Basically, I am excited about the concept of this workshop, but I honestly 
think we've tried to implement it too soon. There are a number of problems 
which need to be addressed: 

1) Lack of teacher training/experience on computers 

- Many teachers are intimidated to try using them with students 

2) Lack of student training/experience on computers 

- Before the computer can be used to teach vocabulary, reading, 
etc., some valuable educational time is inevitably cost when 
the English teacher must turn his/her English class into a 
computer course or a typing course. 

3) Insufficient Equipment 

- Too few computers in schools — often tied up by' business and 
computer classes. Our own computer lab does not have enough 
computers for each student in an English classroom to have 
his/her own computer. The problem becomes even more severe if 
the project requires printing out a draft or information as 
there are even fewer printers available than computers. 

There are also some problems in that some software programs are not 
conducive to fifty-minute periods. Most students cannot "finish up" 
at home and it is often difficult to find them computer time when 
not in class. 




The concept of this workshop jis essential as we rapidly approach the 
21st century. But some attention does need to be paid to the foundation 
of the structure. a i 
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Social Studies , School: 

English/Language Arts District: 

Date: 



DELAWARE COMPUTER INSERVICE PROGRAM 



TEACHER LOG 



Please fill out one teacher log for each instructional plan that you develop and 
implement during phase IV, Attach the log to its corresponding plan and submit 
it to the instructional leader upon completion of the lesson. 



A- PLAN #1 #2 #3 #4 #5 (please circle) 

B. PLEASE PROVIDE THE FOLLOWING INFORMATION ABOUT THIS PLAN: 

1. Title of Plan : 

2. Delaware Curriculum Standard Matched : y 

3. Software Used : 

A. Date Implemented : 

5. Time Implemented : 

6. Number of Students and Grade Level : 

C. PLEASE COMPLETE THE FOLLOWING EVALUATION OF THIS PLAN: 

Rate the extent to which each of the following was accomplished: 
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7. the plan was implemented as written 
Comments ; 

I have access to a computer every three 
weeks; Not possible to do - only 1 disc and 
no printed matter for the program. 



Fully 

4 
71 



42 
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3 
27 



Not 
At All x 



1 
0 



3.68 



BEST COPY AVAILABLE 



Fully 



Not 
At All 



8. the plan objectives were achieved 
Comments : 

Ml students improved scores by end of week; 
After using computer students were very 
enthusiastic about independent work 



4 

65 



3 
32 



2 
3 



1 

0 



3.62 



9. the plan promoted student achievement 
of standard 



4 

43 



54 



2 
3 



1 
0 



3.41 



Comments : 



10. the plan promoted student motivation (i.e., 4 3 2 1 

interest and enthusiasm) 74 26 3^74_ 

Comments ; 

The children loved it (3); Very much so; Fun; They 
were quite enthused up to and beyond the bell; 
They were proud to be able to print their own 
stories . 



11. the plan proved more effective than regular 
instruction 



4 3 2 1 

60 21 16 3 3.39 



Comments : 

Provided variety (2); Because of its novelty, was 
used enthusiastically but did not yield higher 
test scores for my really poor spellers; Helped with 
motivation; Kept interest; More effective and / 
enjoyable; Non-typists were slow in using keyboard. 

12. Do you have any additional suggestions for improving this instructional plan 
or the software used? 

No, it worked well; Children used to be exposed to the computer more often 
and for longer time periods; Would like more; Need more time; More access 
to computer; Having to set up the equipment in 5 minutes before class and 
return all of it was a big inconvenience; One student used a password to 
enter his story - had trouble getting it back. 
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RESULTS (N=3) 



School: 

District: 

Date: 



MATERIALS FEEDBACK FORM (II) 

Directions: Rate the adequacy of the inservice materials (inservice manual and 
resource books) for your use in each of the activities listed belov. Also, space 
is provided under each of these activities for your suggestions/comments. 



PHASE I: ORIENTATION 

How adequate vere the inservice materials for : 
9 Preparatory Activities 
Suggestions/Comments : 



Very 
Adequate _JL 



4 
67 



3 
33 



Not 

Adequate x 



3.67 



o Activity 1: Review inservice goals, 
objectives, and guidelines 



4 
67 



3 
33 



3.67 



Sugges t ions/Commen ts : 



o Activity 2: Explore uses of computer 
in instruction 

2.1: Discuss current teacher experience 4 3 '2 1 

67 33 2 - 67 

Suggestions/Comments : 

As we were discouraged from talking about 
negative experiences, a basically inaccurate 
picture was drawn. 

2.2: Demonstrate software 4 3 2 1 

100 

Sugges t ions/Commen ts : 
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Very Not _ 

Adequate % Adequate _x_ 



• Activity 3: Assess teacher understanding A 3 2 1 2^67 

of computer uses in instruction 

Suggestions/Comments : 

I don't recall much time/discussion being 
spent on this . 



PHASE II: DEMONSTRATION OF PREVIEWING 

Hov adequate vere the inservice materials for: 
• Preparatory Activities 
Suggest ions /Comments : 

Generally because of the lack of necessary 
materials, this whole part of course was 
pretty weak. 

' • Activity 1< Provide a brief overview 
of selecting software and developing 
instruccional plans 

Sugges tions/Commen ts : 

Basically I'm still confused about where 
to locate so f cware. I know what Dover 
High owns. I know that all MECC is 
available. But from whom? Where? 

• Activity 2: Demonstrate an approach to 
selecting software 

2.1: Introduce the demonstration 

Su ggest ions /Comments : 

Social studies titles didn't fit into 
curriculum as expected. 



A 

33 



A 

67 



A 
33 



3 

33 



3 
33 



2 
33 



2 

33 



1 

33 



2.67 



3.33 



3.00 



2.2: Demonstrate Delaware Standards/ 
Software Hatching System 

Sugges tions/Commen ts : 

Standards and software didn't clearly 
match. 



2 
67 



1 

33 



1.67 
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2.3: Review information provided in 
such resource as EPIE and TESS 

Suggestions/Comments : 

I don't recall seeing it in my training 
and got my school's copy after we completed 
Phase III; Need to emphasize these sources 
more . 

2.4: Previev with a group, a selected 
piece of software 

Suggestions/Comments : 

Familiarization with software by demon- 
strator is crucial. 



Very 

Adequate % • 

4 3 
67 



NOt 

Adequate 



4 
33 



3 
67 



2 
33 



2.67 



3.33 



2.5: Decide whether to incorporate soft- 
ware into current instructional plans 



3 

100 



3.00 



Suggestions/Comments : 

Need to encourage taking that extra step 
despite logistical problems. 



2.6: Develop, with the group, a revised 
instructional plan 



4 

50- 



2 
'50 



3.00 



S uggestions/Comments : 

Emphasize computer as a tool, not a replacement for instruction; 
I actually think this step should be omitted. The group plan did 
not really help in devising the independent plan. In retrospect, 
I thi?ik it would have been more helpful for each of us to have 
devised/ implemented our own plan. 
2.7: Review instructional plan(s) against 4 
criteria 

Suggestions/Comments: 



3 

100 



3.00 



• Activity 3: Review preparatory activities 
for Phase III 



4 
33 



3 
67 ' 



3.33 



Suggestions/Comments : 
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PHASE III: DEVELOP AND IMPLEMENT AN INSTRUCTIONAL 
PLAN VITH GUIDANCE 

Hov adequate were the inservice materials for : 

• Activity 1: Select software and develop 
an instructional plan 

Comments : 

Lack of DSCSMS and value terms ("Instructional 
Plan," "Activity," etc.) 

• Activity Share sofvare previews and 
review instructional plan 

Comments : 

People assess according to their success with 
the computers they use. Slight changes can 
make a difference. 

' • Activity 3: Implement instructional plan 
Comments: 



• Activity A: Revise instructional plan 
Comments: - 



• Activity 5: Share implementation 
experience and revised instructional 
plan 

Comments: 



• Activity 6: Review preparatory activities 6 y 33 2 1 3 67 

for Phase IV 

Comments : 

M 



Very 
Adequate 



Not 
Adequate 



4 

33 



3 
33 



2 
33 



3.00 



3 

100 



3.00 



4 

33 



3 
67 



3.33 



4 

33 



3 
67 



3.33 



4 

33 



3 
67 



3.33 



